Finding links between multiple projects
2021-05-04
An experiment to pick up keywords from the most recent one-day update in Scrapbox's Project A and present pages containing those keywords from Project B.
In this case, A is this public project and B is a private project of about 400 pages
If it seems to have a good effect, run it regularly once a day.
context
Diary of 2021-05-04
About Scrapbox private to public transfer
What we did.
Get ✅public's most recently updated pages
✅Extract keywords from its content
Just the one that is explicitly bracketed for now.
You can get it from the API.
✅Find and present ✅from different sources that contain those keywords.
There are two ways to search on your own after exporting to JSON or by hitting the search API.
This time, the search API
How to present
Create a page in private
Because the search results of a private project cannot be made public
There is an option to send a DM in Slack.
✅ this time I printed it and pasted it into Scrapbox by hand
result
(of) good appearance
I thought it was a private memo repository with not much information, but there were some surprisingly interesting pages.
I got some hits from the experiment, like the one where I transcribed a recording of a video conference.
105 keywords extracted from 24 hours of activity
The number of hits found in the search is 11.
The search API yields the hit rows.
There are some not-so-good keywords.
Research, Organization, Scrapbox, Unexplored Jr.
Too many hits.
It is better to display only the information "many hits" if the number of hits exceeds a certain level.
In the original project, it is only connected if both pages are bracketed, which is not a large number.
Three of the four cases were bracketed.
So, if you have a lot of search results, you can only show the number of results instead of the number of results, and bracket that search term instead.
Always do search keyword bracketing.
Rather, "Is there already a link?" and if so, is it better not to search?
impressions
relatively good
There are a number of other projects that seem to have a lot of interesting content dead in the water.
I want to have more than one target project.
additional experiment
I was thinking mainly of my public/private collaboration, so I couldn't publish the results because one of them was private.
I realized I could do both public.
And with the current version, which uses search instead of export, you can target other people's projects without any permission.
interest :
/rashitamemo/interesting(query): 24 hits
detection :
/rashitamemo/found(query): 88 hits
awareness :
/rashitamemo/noticed(query): 10 hits
spontaneous :
/rashitamemo/spontaneous(query): 11 hits
intellectual production :
/rashitamemo/intellectual-production(query): 100 hits
Puppeteer:
/rashitamemo/Puppeteer(query): 1 hits
puppeteer:
puppeteer
GoogleChrome/puppeteer: Headless Chrome Node API
Puppeteer is a Node library which provides a high-level API to control headless Chrome or Chromium over
[https://codeburst.io/a-guide-to-automating-scraping-the-web-with-javascript-chrome-puppeteer-node-js-b18efb9e9921
A Guide to Automating & Scraping the Web with JavaScript (Chrome + Puppeteer + Node JS)]
Heroku:
/rashitamemo/Heroku(query): 1 hits
TODO.txt:
I studied about the task management method "Todo.txt" | Tofu mentality does not collapse
Headless Chrome:
/rashitamemo/Headless Chrome([query https://scrapbox.io/rashitamemo/search/page?q=Headless Chrome]): 1 hits
puppeteer:
GoogleChrome/puppeteer: Headless Chrome Node API
Puppeteer is a Node library which provides a high-level API to control headless Chrome or Chromium over
It can also be configured to use full (non-headless) Chrome or Chromium.
It is thought to be a headless Chrome that can be operated from the command line.
[https://codeburst.io/a-guide-to-automating-scraping-the-web-with-javascript-chrome-puppeteer-node-js-b18efb9e9921
A Guide to Automating & Scraping the Web with JavaScript (Chrome + Puppeteer + Node JS)]
research :
/rashitamemo/research(query): 100 hits
MVP:
/rashitamemo/MVP(query): 1 hits
Blog 2.0 :
fbclid=IwAR3LXwnspP4W9R79YvAjlC760Q62f_kgBMVpUmRP9WmYok8_xWEMVLkjyZM]
(late Edo-period) culture of debauchery :
/rashitamemo/tomatology culture(query): 1 hits
Kurashita output in April 2021 :
Sigotano not tasked with bean theory culture!
campfire (usually large, and for gathering and singing, etc.) :
/rashitamemo/campfire(query): 1 hits
June 2019 Idea Line :
[I want to spread more people to use the most powerful time management technique that saved my life, "Task Shoot Time Technique" - CAMPFIRE (Campfire) https://camp-fire.jp/projects/view/162572
gated community :
/rashitamemo/gated-community(query): 3 hits
gated community :
gated community
A gated community is a residential area with a gate (gate) and a wall around the perimeter to restrict access to the property by non-residents, thereby preventing through traffic and improving security.
The concept of a gated community itself is not new; there have been concessions and U.S. military houses in the past, and it is merely a redefinition of the term. In Japan, it is also referred to as a gated town1 or gated community2. `
Walled Garden Model :
→ gated community
gated content :
→ gated community
organization :
/rashitamemo/organization(query): 84 hits
decline :
/rashitamemo/decline(query): 8 hits
elimination :
/rashitamemo/exclusion(query): 14 hits
Scrapbox:
/rashitamemo/Scrapbox(query): 100 hits
Updated to account for the case where the output project and the project to be searched are different.
The [1] in the quote from rushitamemo has become a search keyword.
Should it be ~? I fixed it by hand for now.
The output of a machine should not be fixed by hand and maintained for a long period of time.
I'm only doing it now because it's an experiment.
It's confusing the links for this project, so I'll eventually kick it out to gist or something.
Hopefully, if nothing is done, it will be overwritten after a day and disappear.
Any useful links should be noted on a separate page
interest :
/shokai/interesting(query: 1 hits)
/shokai/remote-work:.
(Do we need to be funny about how we do things?)
detection :
/shokai/found(query: 25 hits)
awareness :
/shokai/notice(query: 3 hits)
/shokai/not a warehouse where dead texts are kept:.
It's hard to realize, but it's a complete waste of time and effort, because the tedious procedure somehow gives you a sense of doing your job and a mysterious sense of satisfaction.
/shokai/tempura-takao:.
I noticed that there is also a store in Joynas, Yokohama, and I have been going there about once every two weeks since then.
/shokai/Milkcocoa:
I realized that I had no motivation to create libraries for devices I didn't use, and I resolved to only create things I would use from then on.
Heroku:
/shokai/Heroku(query: 39 hits)
Headless Chrome:
/shokai/Headless Chrome([query https://scrapbox.io/shokai/search/page?q=Headless Chrome]: 1 hits)
/shokai/roppongi.js#1:
Headless Chrome library was created and got 2000stars by yujiosaka.
https://speakerdeck.com/yujiosaka/hetudoresuchromedekurorawozuo-tutahou-falsehua-1
https://github.com/yujiosaka/headless-chrome-crawler
research :
/shokai/research(query: 26 hits)
1:
/shokai/1(query: 100 hits)
2:
/shokai/2(query: 100 hits)
organization :
/shokai/organization(query: 14 hits)
decline :
/shokai/decline(query: 1 hits)
/shokai/Mob Programming:
The decline of management without technology and what to do about it - The Methodologist's Blog
elimination :
/shokai/exclusion(query: 7 hits)
/shokai/Internet language can be rough because of the short term nature of the recommendation service:: /shokai/Internet language can be rough because of the short term nature of the recommendation service.
Eliminate all features that stimulate approval.
/shokai/eliminate all functions that lead to the stimulation of the need for approval:.
Eliminate all features that stimulate approval.
/shokai/wiki never works if you use the URL of the page as an ID:.
I want to eliminate all the elements that would spawn a lackey/administrator within WiKi and have to ask before I edit.
/shokai/press-release:: [/shokai/press-release
This and the idea of [Eliminate all features that stimulate approval.
/shokai/I lose my reason when I look at a human face:.
I use Scrapbox to eliminate those elements that are obscured by the appearance of real human faces.
/shokai/want to reduce elements that block thinking:: [/shokai/want to reduce elements that block thinking
Like most editors, I've eliminated switching between edit and view modes because it's only a brake on my thinking, and if there are other toolbars for text decoration, I spend more time fiddling with the font size than writing text.
It is thoroughly eliminated because the time is more and the acceleration is lost.
/shokai/media whose memory is reset every time:: [/shokai/media whose memory is reset every time
I especially recommend scrapbox among the wikis because it allows you to write at the speed you think and eliminates the elements that get in the way of your thinking.
Scrapbox:
/shokai/Scrapbox(query: 100 hits)
Hatena Diary :
/shokai/Hatena Diary(query: 1 hits)
/shokai/Internet language can be rough because of the short term nature of the recommendation service:: /shokai/Internet language can be rough because of the short term nature of the recommendation service.
I think the exchange of opinions should be done on Hatena Diary or Hatena Blog.
dotenv:
/shokai/dotenv(query: 2 hits)
/shokai/dotenv:
dotenv
http://npmjs.org/packages/dotenv
Can be loaded with require('dotenv').load().
/shokai/dotenvify:
dotenvify
https://www.npmjs.com/package/dotenvify
Transform of Browserify that executes dotenv just before executing [envify
$ browserify -t dotenvify src/index.js -o build/index.js
When .env is missing, dotenv gives warnings
It can be suppressed with the silent option, but dotenvify does not have the ability to pass options to dotenv
Added functionality: https://github.com/shokai/dotenvify/tree/dotenv-options
$ npm i shokai/dotenvify#dotenv-options -save
$ browserify -t dotenvify --silent --path local.env src/index.js -o build/index.js
Secure:
/shokai/Secure(query: 1 hits)
bookmarklet for pasting pull requests from /shokai/Github to Scrapbox:: bookmarklet
Use secure protocol instead of http #79
[https://github.com/shokai/tw/pull/79 Use secure protocol instead of http by sachin21 · Pull Request
HttpOnly:
/shokai/HttpOnly(query: 2 hits)
/shokai/Heroku to do Socket.IO:.
Version=1; Expires=Tue, 15-Nov-2016 11:51:24 GMT; Max-Age=86400; Domain=staging.scrapbox.io; Path=/; HttpOnly
/shokai/sticky-session:
Version=1; Expires=Tue, 15-Nov-2016 11:51:24 GMT; Max-Age=86400; Domain=staging.scrapbox.io; Path=/; HttpOnly
requests:
/shokai/requests(query: 1 hits)
/shokai/login with Passport to GHE operated with ole SSL certificate:: /shokai/login with Passport to GHE operated with ole SSL certificate:: [/shokai/login with Passport to GHE operated with ole SSL certificate
[Adding ability to specify an agent for OAuth2 requests by PhilipSkinner · Pull Request #324 · ciaranj
impressions
It's nice to see other people's presentations on technical keywords that I'm interested in and have written down.
We all think we are using the same Japanese, but there are differences in the choice of words we use to express our thoughts.
In my project, I use the word "exclusion" to refer to the removal of a person from a group of people (this is not my word either, but something I wrote down after seeing other people's expressions).
On the other hand, shokai is used to refer to "removing harmful features or functions from software.
By comparison Notice the difference.
If you look at the keywords displayed and tweak the page in various ways, of course it will be an "updated page" tomorrow, and the same thing will be displayed.
This must be boring.
Remember what you put out once and don't put it out for a week to a month, and so on.
source https://github.com/nishio/scbot/blob/0.2/scbio.py#L113
---
This page is auto-translated from /nishio/複数のプロジェクト間のリンクを見つける using DeepL. If you looks something interesting but the auto-translated English is not good enough to understand it, feel free to let me know at @nishio_en. I'm very happy to spread my thought to non-Japanese readers.